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Amendments to the Claims: 

Following is a complete listing of the claims pending in the application, as amended, 
which replaces all prior versions and listings of claims in the application: 

1 . (Currently Amended) A method for a computer system to use human assistance in 
performing tasks, the method comprising: 

automatically and under control of a first computer system, causing a task to be 
performed by, 

identifying a first and a second subtask of the task vJickio the s<w viht^L 
oh e- -jve* h uv*v^ntiet>; 

retrieving information about past quality of results of one or more humans when 
previously performing subtasks other than the first subtask; 

using the retrieved past quality information to facilitate performance of the first 
subtask by one or more humans, the facilitating of the performance of the first subtask including, 
identifying one or more required capabilities for performance of the first 

subtask; 

dispatching the first subtask to a remote second computer system of a first 
human for performance by the first human, the first human identified as being one of one or 
more humans who have capabilities that satisfy the required capabilities for the first subtask, the 
retrieved past quality information including past quality information for the first human when 
previously performing multiple subtasks other than the first subtask, the past quality information 
for the first human being at least part of the capabilities of the first human that satisfy the 
required capabilities for the first subtask; and 

receiving a first result from the first human via the second computer 
system, the first result generated by performance of the first subtask by the first human; 

providing payment to the first human for the performance of the first subtask; and 
generating a result for the task based at least in part on the first result. 
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2. (Previously Presented) The method of claim 1 wherein the identifying by the first 
computer system of the first and second subtasks of the task includes decomposing the task into 
at least the first and second subtasks. 

3. (Previously Presented) The method of claim 1, wherein 

the method further comprises the first computer system dispatching said second subtask 
to a third computer system of a second human for performance by the second human, and the 
first computer system receiving a second result from the second human via the third computer 
system for said second subtask; and 

the first computer system further bases its generation of the result for said task on said 
second result. 

4. (Previously Presented) The method of claim 3, wherein 

said task further comprising a third subtask, and the method further comprises the first 
computer system receiving and performing said third subtask producing a third result; and 

the first computer system further bases its generation of the result for said task on said 
third result. 

5. (Previously Presented) The method of claim 1, wherein 

the method further comprises the first computer system performing said second subtask 
producing a second result; and 

the first computer system further bases its generation of the result for said task on said 
second result. 

6. (Previously Presented) The method of claim 1, wherein said first human is one of 
college educated, at most high school educated, at most elementary school educated, and not 
formally educated. 

7. (Original) The method of claim 1, wherein said subtask is one of text, speech, 
sound, and images related operations. 
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8. (Original) The method of claim 1, wherein said result is one of text, numbers, 
tuples, and sound. 

9. (Original) The method of claim 1, wherein said task is one of text classification, 
image comparison, image processing, speech comparison, speech recognition, conversion of 
speech into text, and comparison of music samples. 

10. (Previously Presented) The method of claim 1, wherein said task is associated 
with multiple attributes related to performance of said task, the attributes including an accuracy 
attribute, a security attribute, a timeout attribute, a maximum time spent attribute, a maximum 
cost per task attribute, and a maximum total cost attribute, and wherein the identifying of the one 
or more humans, the dispatching of the first subtask, and the generating of the result for said task 
are performed in a manner to reflect the multiple associated attributes. 

11. (Original) The method of claim 1, wherein said task is associated with one or 
more attributes, and said attributes include an accuracy attribute. 

12. (Currently Amended) A method for a computing system to use human assistance 
in performing tasks, the method comprising: 

receiving an indication of a first subtask to be performed, the first subtask having one or 
more associated criteria related to performance v - ^ v 

sending an indication of the first subtask to multiple humans to each perform the first 
subtask, each of the humans being identified as being capable of satisfying at least some of the 
associated criteria for the first subtask; 

from each of at least some of the multiple humans, receiving a result of performance of 
the first subtask by the human; and 

automatically facilitating generation of a final result for the first subtask by, 

determining that the received results include a common result that was received 
from each of a selected number of humans, the selected number of humans being greater than 1 
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and based on at least one of a majority of the at least some humans and of at least a specified 
number N of two or more humans; and 

selecting the common received result as the final result for the first subtask. 

13. (Previously Presented) The method of claim 12 further comprising tracking 
accuracy of the at least some humans based at least in part on whether a result received from a 
human is the common result. 

14. (Previously Presented) The method of claim 12 wherein the first subtask is part of 
a task, and wherein the method further comprises automatically generating a result for the task 
based at least in part on the selected result for the first subtask and on the accuracy of the at least 
some humans who provided the selected result. 

15. (Original) The method of claim 1, wherein said task is associated with one or 
more attributes including a security attribute, and said security attribute comprises a selection of 
one of a "strict" security level, a "lax" security level, and "no" security level. 

16. (Previously Presented) The method of claim 1, wherein said task is associated 
with one or more attributes that include a "maximum time" attribute specifying a maximum 
amount of time to be spent by an assigned human to perform said first subtask. 

17. (Original) The method of claim 1, wherein said task is associated with one or 
more attributes, and said attributes include a maximum cost per task attribute. 

18. (Original) The method of claim 1, wherein said task is associated with one or 
more attributes, and said attributes include a maximum total task cost attribute. 

19-31. (Canceled) 
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32. (Currently Amended) An apparatus comprising: 

a storage medium having stored therein a plurality of programming instructions that are 
machine executable, wherein when executed, said instructions operate to: 

receive indications of a first and a second subtask of a task, the first subtask 
associated with one or more capabilities to be used in performance of the first subtask and having 
associated compensation for performance of the first subtask - ^ s v * s v\ 

dispatch information about the first subtask to a remote computer system of a first 
human for performance by said first human of the first subtask, the first human identified as 
having one or more capabilities that satisfy the associated capabilities for the first subtask, the 
dispatched information including an indication of the associated compensation for performance 
of the first subtask; 

receive a first result from the remote computer system based on performance of 
the first subtask by said first human; 

retrieve information about past accuracy of said first human in previously 
performing subtasks; and 

generate a result for said task based at least in part on said first result and at least 
in part on the past accuracy information for said first human; and 

a processor coupled to said storage medium to execute said instructions. 

33. (Previously Presented) The apparatus of claim 32, wherein the receiving of the 
indications of the first and second subtasks of the task includes decomposing the task into at least 
the first and second subtasks. 

34. (Previously Presented) The apparatus of claim 32, wherein said instructions, when 
executed, further operate to dispatch information about said second subtask to a distinct remote 
computer system of a second human for performance by the second human, receive a second 
result from the second human via the distinct remote computer system for said second subtask, 
and generate the result for said task further based on said second result. 
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35. (Original) The apparatus of claim 34, wherein said instructions, when executed, 
further operate to decompose said task into at least said first, said second, and a third subtask; 
perform said third subtask producing a third result; and generate the result for said task further 
based on said third result. 

36. (Original) The apparatus of claim 32, wherein said instructions, when executed, 
further operate to perform said second subtask producing a second result, and generating the 
result for said task further based on said second result. 

37. (Previously Presented) The apparatus of claim 32, wherein said first human is one 
of college educated, at most high school educated, at most elementary school educated, and not 
formally educated. 

38. (Previously Presented) The apparatus of claim 32, wherein the dispatched 
information about said first subtask includes speech to be reviewed by the first human. 

39. (Previously Presented) The apparatus of claim 32, wherein said result is one of 
numbers and tuples. 

40. (Canceled) 

41. (Previously Presented) The apparatus of claim 32, wherein said task is associated 
with attributes that include multiple of an accuracy attribute, a security attribute, a timeout 
attribute, a maximum time for a task attribute, a cost per task attribute, and a maximum task cost 
attribute. 

42. (Previously Presented) The apparatus of claim 41, wherein said instructions, when 
executed, further operate to dispatch information about the first subtask to multiple additional 
humans to perform said first subtask, and said accuracy is based at least in part on receiving 
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results in agreement from multiple of the humans to whom the information about the first 
subtask is dispatched. 

43. (Original) The apparatus of claim 42, where said instructions, when executed, 
further track the accuracy of the humans. 

44. (Original) The apparatus of claim 42, where said instructions, when executed, 
further take into consideration the accuracy of the humans when generating the result. 

45. (Original) The apparatus of claim 41, wherein said security includes one of a strict 
security level, a lax security level, and no security. 

46. (Previously Presented) The method of claim 1 wherein the first computer system 
is a task server system that is part of a distributed hybrid computer/human computation 
arrangement, and wherein the first human is one of numerous humans remote from the task 
server system who each use distinct client computing devices to act as nodes of the distributed 
hybrid computer/human computation system. 

47. (Previously Presented) The method of claim 46 wherein the distributed hybrid 
computer/human computation arrangement further includes one or more distinct coordinating 
server computing systems remote from the task server system such that the dispatching of the 
first subtask by the task server system includes sending information to at least one of the 
coordinating server computing systems that includes an indication of the first subtask and of the 
identified required capabilities, and wherein one of the coordinating server computing systems 
coordinates performance of the first subtask by identifying the first human as having capabilities 
that satisfy the identified required capabilities and by sending to the second computer system of 
the first human an indication of the first subtask to be performed. 

48. (Previously Presented) The method of claim 1 wherein the first computer system 
is a coordinating server computing system that identifies the first and second subtasks of the task 
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by receiving information about the first and second subtasks from a remote task server system, 
wherein the coordinating server computing system causes the result to be generated for the task 
based at least in part on the first result by sending the received first result to the remote task 
server system, and wherein the first human is one of numerous humans who each use distinct 
client computing devices to act as nodes available to the coordinating server computing system. 

49. (Previously Presented) The method of claim 1 wherein the dispatching by the first 
computer system of the first subtask to the remote second computer system is performed using a 
defined application programming interface ("API"). 

50. (Previously Presented) The method of claim 1 wherein the dispatching by the first 
computer system of the first subtask to the remote second computer system is performed by 
programmatically sending one or more messages from the first computer system to the remote 
second computer system. 

5 1 . (Previously Presented) The method of claim 1 wherein the dispatching by the first 
computer system of the first subtask to the remote second computer system of the first human 
includes providing an indication to the first human of the payment to be provided for 
performance of the first subtask if the first human chooses to perform the first subtask. 

52. (Previously Presented) The method of claim 1 wherein the providing of the 
payment to the first human is performed in response to the receiving from the first human of the 
first result from the performance of the first subtask. 

53. (Previously Presented) The method of claim 1 wherein the required capabilities of 
the human for performance of the first subtask include an ability to speak a specified language. 

54. (Previously Presented) The method of claim 1 wherein the required capabilities of 
the human for performance of the first subtask include an ability to hear. 
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55. (Previously Presented) The method of claim 1 wherein the required capabilities of 
the human for performance of the first subtask include a specified degree of historical accuracy 
by the human when performing subtasks. 

56. (Previously Presented) The method of claim 1 wherein the payment provided to 
the first human for the performance of the first subtask is further based in part on quality of the 
performance of the first subtask. 

57. (Previously Presented) The method of claim 1 wherein the payment provided to 
the first human for the performance of the first subtask is further based in part on cumulative 
contributions of the first human. 

58. (Previously Presented) The method of claim 1 wherein the payment provided to 
the first human for the performance of the first subtask is further based in part on a prior 
agreement with the first human. 

59. (Previously Presented) The method of claim 1 wherein the first compensation 
level is a first monetary amount, and wherein an amount of the payment provided to the first 
human for the performance of the first subtask is the first monetary amount. 

60. (Previously Presented) The method of claim 59 wherein the second subtask has a 
distinct second monetary amount of compensation associated with performance of the second 
subtask, and wherein the method further comprises providing payment of the second monetary 
amount of compensation to a second human for performance of the second subtask. 

61. (Previously Presented) The apparatus of claim 32 wherein said instructions, when 
executed, further operate to provide the associated compensation to the first human for the 
performance of the first subtask based in part on the receiving of the first result. 
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62. (Previously Presented) The method of claim 14 wherein the receiving of the 
indication of the first subtask includes automatically decomposing the task into at least the first 
subtask and a second subtask. 

63. (Previously Presented) The method of claim 12 wherein the first subtask further 
has associated compensation for performance of the first subtask, and wherein the method further 
comprises facilitating providing of the associated compensation to one or more of the humans 
who provided the common result. 

64. (Previously Presented) The method of claim 12 wherein the multiple humans to 
whom the indication of the first subtask is sent include a number Nl of the multiple humans, 
such that N and Nl are integers and Nl is greater than N. 

65. (Previously Presented) The method of claim 12 wherein the at least some humans 
from whom the results of performance of the first subtask are received include a number Nl of 
multiple humans, such that N and Nl are integers and Nl is greater than N. 

66. (Previously Presented) The method of claim 12 wherein the facilitating of the 
generation of the final result for the first subtask includes generating the final result so as to have 
a desired accuracy by automatically determining the selected number of humans for that desired 
accuracy. 

67. (Previously Presented) The method of claim 12 wherein the receiving of the 
indication of the first subtask includes receiving an indication of a desired accuracy for the final 
result, and wherein the facilitating of the generation of the final result is performed such that the 
final result has the indicated desired accuracy. 

68. (Previously Presented) The method of claim 12 wherein the facilitating of the 
generation of the final result for the first subtask includes receiving an indication to use a 
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majority governs policy and automatically determining the selected number of humans to be a 
number corresponding to the majority of the at least some humans. 

69. (Previously Presented) The method of claim 12 wherein the facilitating of the 
generation of the final result for the first subtask includes receiving an indication to use the 
specified number N for the selected number of humans, and automatically determining the 
selected number of humans to be N. 

70. (Previously Presented) The method of claim 12 wherein the facilitating of the 
generation of the final result for the first subtask includes, before the determining that the 
received results include the common result from each of the selected number of humans, 
determining that a first group of multiple results received from multiple of the humans for the 
first subtask do not include a common result that was received from each of the selected number 
of humans, and sending indications of the first subtask to additional humans in order to obtain 
the common result from each of the selected number of humans. 

71. (Previously Presented) The method of claim 12 wherein the common result 
received from the selected number of humans includes multiple results that are in agreement. 

72. (Previously Presented) The method of claim 12 wherein the common result 
received from the selected number of humans includes multiple copies of a single result. 

73. (Previously Presented) The method of claim 12 wherein the first subtask involves 
image processing. 

74. (Currently Amended) % ^ n ^ N N ^ , 




'•!•,-'•, iv ( « ''ak J os\: - it :vl i\\ \ ^ •, wherein the first subtask involves speech 
comparison;.- 
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75-76. (Canceled) 

77 (C urrcntly Amended) v e\ ^ s \ \ v s 

so v v wherein the first subtask involves comparison of 

music samples:, 



N of two or more humans; and 

s ^ v ■> ^ N rosish jMbo fntA Milt for the fust Mtbtask 
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78. (Previously Presented) The method of claim 12 wherein the first subtask involves 
text classification. 

79. (Canceled) 

80. (Previously Presented) The method of claim 1 wherein the past quality 
information for the first human when previously performing subtasks other than the first subtask 
reflects a past accuracy of the first human in performing those other subtasks. 

81. (Previously Presented) The method of claim 80 further comprising, after the 
receiving of the first result from the first human, determining an accuracy of the first result and 
updating the past accuracy information for the first human to reflect the determined accuracy. 

82. (Previously Presented) The method of claim 81 wherein the determining of the 
accuracy of the first result includes obtaining one or more results that are each generated by 
performance of the first subtask by a human other than the first human, and using the obtained 
one or more results as part of the determining of the accuracy of the first result. 

83. (Previously Presented) The method of claim 82 wherein the using of the obtained 
one or more results as part of the determining of the accuracy of the first result includes 
weighting each of the obtained one or more results based at least in part on retrieved past 
accuracy information for the human whose performance generated the result. 

84. (Previously Presented) The method of claim 1 wherein the generating of the result 
for the task is based at least in part on the past quality information for the first human. 

85. (Previously Presented) The method of claim 1 wherein the payment provided to 
the first human is based at least in part on the past quality information for the first human. 
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86. (Previously Presented) The method of claim 1 wherein the dispatching of the first 
subtask to the remote second computer system of the first human includes providing an 
indication to the first human of compensation associated with performance of the first subtask. 

87. (Previously Presented) The method of claim 1 wherein the payment provided to 
the first human is based at least in part on quality of the first result. 

88. (Previously Presented) The method of claim 1 wherein the first human has at most 
an average level of skill with respect to performing the first subtask. 

89. (Previously Presented) The method of claim 1 wherein the first human is not an 
expert with respect to performing the first subtask. 

90. (Previously Presented) The method of claim 1 wherein the first subtask involves 
an activity that is straightforward for a typical human but is difficult for a computer system. 

91. (New) A storage medium having stored therein a plurality of programming 
instructions that are executable to cause a first computing system to perform a method 
comprising: 

receiving indications of a first and a second subtask of a task, the first subtask associated 
with one or more capabilities to be used in performance of the first subtask and having associated 
compensation for performance of the first subtask, wherein said task is one of speech 
comparison, speech recognition, conversion of speech into text, and comparison of music 
samples; 

dispatching information about the first subtask to a remote computer system of a first 
human for performance by said first human of the first subtask, the first human identified as 
having one or more capabilities that satisfy the associated capabilities for the first subtask; 

receiving a first result from the remote computer system based on performance of the first 
subtask by said first human; 
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retrieving information about past accuracy of said first human in previously performing 
subtasks; and 

generating a result for said task based at least in part on said first result and at least in part 
on the past accuracy information for said first human. 

92. (New) The storage medium of claim 91, wherein said instructions, when 
executed, further operate to decompose the task to be performed into at least the first and second 
subtasks. 

93. (New) The storage medium of claim 91, wherein said instructions, when 
executed, further operate to dispatch an indication of said second subtask to a computing system 
for performance by a second human, receive a second result from the second human for said 
second subtask, and generate the result for said task further based on said second result. 

94. (New) The storage medium of claim 93, wherein said instructions, when 
executed, further operate to decompose said task into at least said first, said second, and a third 
subtask; perform said third subtask producing a third result; and generate the result for said task 
further based on said third result. 

95. (New) The storage medium of claim 91, wherein said instructions, when 
executed, further operate to automatically perform said second subtask producing a second 
result, and generate the result for said task further based on said second result. 

96. (New) The storage medium of claim 91, wherein said first human is one of 
college educated, at most high school educated, at most elementary school educated, and not 
formally educated. 

97. (New) The storage medium of claim 91, wherein said first subtask is one of text 
and speech. 
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98. (New) The storage medium of claim 91, wherein said result is one of text, 
numbers, and tuples. 

99. (New) The storage medium of claim 91, wherein said task is associated with 
attributes that include an accuracy attribute, a security attribute, and a timeout attribute. 

100. (New) The storage medium of claim 99, wherein said instructions, when 
executed, further operate to dispatch said first subtask to Nl - 1 additional humans to perform 
said first subtask, and wherein said accuracy attribute includes an indication of one of majority 
governs and at least N2 agreed results, with N2 and Nl being integers with Nl greater than N2. 

101. (New) The storage medium of claim 99, wherein said security attribute includes 
an indication of one of a strict security level, a lax security level, and no security level. 

102. (New) The storage medium of claim 91, wherein said task is associated with one 
or more attributes, wherein the attributes indicate a maximum time to be spent on a task, a 
maximum cost to incur per task, and a maximum total cost for the task. 
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